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ABSTRACT effects of the availability of behaviotally 

stated objectives on the learning process »ere studied- It vas 
hypothesized that objectives would serve as orienting 
would help students to attend to, process, and organize relevant 
asoects of displayed infornation in accordance with stated 
Ib^ctiveL ThLefore, the presentation of objectives was expected to 
reduce the nuaber of examples and the amount of o 

learn the task, to facilitate performance on transfer retrieval 

crUerlon i^asires, and to reduce the "|!°^J,*cal 

reasoning abilities. The learning task consisted of „„„ 

imaginary science called the science of Xenograde systems (see EH 
053> It was presented by an IBM 1500/1800 computer-assisted 
?nstru“tiorsyste« to 130 introductory educational psychology and 
science education students. Subjects were randomly 

exannle-only group, an objective-example group, a rule-example group, 
or'n ^jecL?e-rule-example group. A significant rule ««ect was 
found in that the rule groups needed less time and fewer example 
than^the no-rule groups. Objectives significantly reduced the number 
of examples required to learn the task, but did not reduce 
latency. Objectives and/or rules also effected a reduction in e 
requirement for reasoning ability. (Author/JY) 
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13 ABSTRACT 



The purpose of this study was to investigate what effects the availability of 
behaviorally stated objectives would have on the learning process. It was hypothe- 
sized that objectives would serve as orienting stimuli which dispose the student to 
attend to, process, and organize relevant aspects of displayed information in ac- 
cordance with the stated objectives. Therefore, the presentation of objectives was 
expected to reduce the number of examples and amount of time required to learn the 
task, facilitate performance on transfer retrieval criterion measures, and reduce the 
requirements for memory and reasoning abilities. 

The learning task consisted of a hierarchical imaginary science called the 
Science of Xenograde Systems which was presented by an IBM 1500/1800 Computer-Assisted 
Instruction System to 130 introductory educational psychology and science education 
students. 

Before learning the task , all Ss were given a battery of six cognitive ability 
tests comprised of three existing published tests and three task-relevant tests 
developed for this study. The tests were selected to measure abilities postulated to 
have specific relationships to task performance. 

The Ss were randomly assigned to an Example-Only group, an Objective-Example ^ 
group, a ¥ule-Example group, and an Objective-Rule-Example group. All £s were required 
to meet a minimum criterion performance at each level of the task before proceeding to 
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13. Abstract (cont'd) 



A significant rule effect was found on the number of examples and 
the total time required to complete the task with the rule groups needing 
less time and fewer examples than the no-rule groups. Objectives signi- 
ficantly reduced the number of examples required to learn the task, but 
they did not reduce total latency. An analysis of the component latency 
measures revealed that objectives either increased or had no effect on 
display latency but significantly reduced test-item-response latency. 
Apparently the objective treatments affected the efficiency and effec- 
tiveness of the S's information processing and thereby facilitated his 
performance on t^e criterion-test items based on the objectives. 



No significant differences were found between treatments on the post 
or retention tests, but a significant rule effect in favor of the rule 
groups was found on the transfer test. 

The battery of cognitive tests was factor analyzed, and a two- 
factor varimax solution was obtained yieldin' the factors of reasoning 
and associative memory. Regression analyses of the individual ability 
test scores, factor scores, and the criterion measures were conducted. 
Significant ability by treatment interactions were obtained using test- 
item-response latency as criterion and reasoning factor scores, plus 
individual reasoning tests as covariables. Reasoning had a high nega- 
tive relationship to test-item-response latency for £s in the Example- 
Only group, but this relationship was significantly smaller for £s in 
the remaining groups. Therefore, the presentation of objectives and/or 
rules effected a reduction in the requirement for reasoning ability. 

On the basis of the results of this study it was concluded that 
objectives have orienting and organizing effects which dispose student.s 
to attend to, process, and structure relevant information in accordance 
with the given objectives. 
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ABSTRACT 



The purpose of this study was to investigate what effects the 
availability of behaviorally stated objecti^ es would have on the learning 
process. It was hypothesized that objectives would serve as orienting stim- 
^^uli which dispose the student to attend to, process, and organize relevant 
aspects of displayed information in accordance with the stated objectives. 
Therefore, the presentation of objectives was expected to reduce the num- 
ber of examples and amount of time required to learn the task, facilitate 
performance on transfer retrieval criterion measures, and reduce the require- 

ments for memory and reasoning abilities. 

The learning task consisted of a hierarchical imaginary science 
called the Science of Xenograde Systems which was presented by an IBM 1500/ 

1800 Computer-Assisted Instruction System to 130 introductory educational 

psychology and science education students. 

Before learning the task, all Ss were given a battery of six cogni- 
tive ability tests comprised of three existing published tests and three task- 
relevant tests developed fox- this study. The tests were selected to measure 
abilities postulated to have specific relationships to task performance. 

The Ss were randomly assigned to an Example-Only group, an Objec 
tive-Example group, a Rule-Example group, and an Objective-Rule-Example group. 
All Ss were required to meet a minimum criterion performance at each level 

of the task before proceeding to the next level. 

A significant rule effect was found on the number of examples and 

the total time required to complete the task with the rule groups needing less 
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time and fewer examples than the no-rule groups. Objectives significantly 
reduced the number of examples required to learn the task, but they did not 
reduce total latency. An analysis of the cornponent latency measures revealed 
that objectives either increased or had no effect or. display latency but sig- 
nificantly reduced test-item-responj^e latency. Apparently the objective 
treatments affected the efficiencj -.id effectiveness of the £’s information 
processing and thereby facilit-.ed his performance on the criterion-test 
items based on the objectives. 

No significant differences were found between treatments on the 
post or retention tests , but a significant rule effect in favor of the rule 
groups was found on the transfer test. 

The battery of cognitive tests was factor analyzed, and a two- 
factor varimax solution was obtained yielding the factors of reasoning and 
associative memory. Regression analyses of the individual ability test 
scores, factor scores, and the criterion measures were conducted. Signifi- 
cant ability by treatment interactions were obtained using test-item-response 
latency as criterion and reasoning factor scores, plus individual reasoning 
tests as covariables. Reasoning had a high negative relationship to test- 
item“response latency for Ss in the Example-Only group, but this relationship 
was significantly smaller for £s in the remaining groups. Therefore, the 
presentation of obj actives and/or rules effected a reduction in the require- 
ment for reasoning ability. 

On the basis of the results of this study it was concluded that 
objectives have orienting and organizing effects which dispose students to 
attend to, process, and structure relevant information in accordance with 
the given objectives. 
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INTRODUCTION 



Statement of the Problem 

It seems that even though educational psychologists (Bobbitt, 
1924; Tyler, 1951; Bloom, 1956) had been stressing the need for precise 
statements of instructional objectives for many years, it was not until 
Mager (1961) published his book on preparing objectives that the educa- 
tional community started to take instructional objectives seriously. 

Since Mager 's book many people have mounted the bandwagon and filled 
the literature with articles extolling the virtues of instructional 
objectives. However, it is disappointing to find that most of these 
articles merely rehash what Mager had previously stated. A few have 
made additional contributions. Gagne and his collegues (Gagne, 1962; 
Gagne 6 Paradise, 1961) have proposed a method of hierarchical task 
analysis based upon the precise statement of instructional objectives 
while Popam (1969) has set up a national clearing house for behaviorally 
stated objectives at UCLA. There are also those (Eisner, 1967a; Ebel, 
1967; Kliebard, 1968; Jackson 6 Belford, 1965) who question the value of 
objectives and feel they might actually be a hindrance to the design of 
instruction. After an interchange of views in the literature, Eisner 
(1967b) responde4 to his critics by pointing out that the contribution 
of educational objectives to curriculum construction, teaching, and 
learning is an empirical problem, while most articles that have been 
written are merely logical arguments. He further claims that the little 
research that has been done is at best inconclusive. 

1 
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The purpose of this study was to investigate what effects the 
presentation of behavioral objectives would have on the learning process. 
Specifically, this study was conducted to further clarify: 1) how the 
presentation of objectives would affect Ss* performance on criteria 
measures, 2) how other task characteristics would vary the effects of 
presenting objectives, and 3) how individual aptitudes interact with 
the presentation or non-presentation of objectives. 

The literature relevant to these issues incorporates the over- 
lapping areas of behavior objectives, task analysis, and aptitude by 
treatment interactions. 

The Effect of the Presentation of Objectives 

Most of the studies which investigated the effects of objectives 
were concerned with the specification of objectives to aid the instruct- 
ional designer or the teacher. Only those studies which have investigated 
the effects of presenting objectives to the students as part of their in- 
struction will be reviewed here. Mager and McCann (1961) conducted a 
study using engineers in an industrial training situation. One group of 
^s were given a specific statement of the training objectives and then 
allowed to instruct themselves by any means or sequence they desired. 

The second group of Ss were allowed to select the content in accordance 
with a self assigned sequence. The third group of Ss received a sequence 
of content controlled by the instructor. The results of this study showed 
that the training time for the group given the objectives and allowed to 
instruct themselves was reduced by as much as 65 percent without a losi> 
in performance. Because of the lack of careful control of conditions in 
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this study, it is not possible to conclude that this draniatic effect was 
solely due to the presentation of objectives to the students. However the 
results do suggest that the presentation of objectives may be a great tool 

for the individualization of instruction. 

Mager and Clark (1963) cited a study (Allen 6 McDonald, 1963) 
where £s were taught how to play a game by one of two instructional methods. 
One group was taught by a linear program while the second group was given 
a list of objectives and allowed to ask the instructor questions. The 
group given the objectives and allowed to ask questions learned the game 
in half the time required by the linearly sequenced group. However, the 
linear program group’s terminal performance was slightly better than that 
of the objective group. The effect of objectives fo\ind in this study is 
also confounded because the objective group was not given the same se- 
quence of content as the linear program group. 

Task Analysis 

It cannot be assumed that any effect foxand by presenting 
objectives to students learning one task will be replicated on different 
tasks. It is, of course, impossible in any single study to replicate an 
effect on all possible tasks. A more economical approach would be to 
analyse a given type of task in an attempt to hypothesize what different 
effects would be expected from differences in task characteristics. Given 
a learning task which contains objectives as part of the task, one can ask 
how the availability of the objectives will affect the student's learning 
activities or information processing? It was hypothesized that objectives, 
presented to the student before the material to be learned, would serve as 
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orienting stimuli which dispose the student to attend to and process, 
relevant aspects of the material. Thus objectives were hypothesized to 
result in saae of those behaviors termed by Rothlcopf (1965) as "mathe- 
magenic behaviors.”. Much research has been done concerning the effects 
of other orienting stimuli such as formal prompts (Anderson 6 Faust, 1967) 
and questions (Frase, 1969). However, in addition to having a selective 
or focusing effect analogous to that found with pre-questions (Frase, 1969), 
it was further hypothesized that objectives would also affect the way the 
learner organizes the material to which he attends. This organization 
should affect the way the material is stored and its availability for 
different types of retrieval. Guilford (1968) distinguishes between two 
types of retrieval: retention retrieval and transfer retrieval. Reten- 

tion retrieval is merely a playback of what is currently in storage while 
transfer retrieval is the selection of relevant material that will assist 
in the solution of a problem in a different context. It was hypothesized 
that objectives could affect S's cognitive organization of information so 
as to aid retention retrieval and/or transfer retrieval depending on how 
the objective was formulated. For example the objective: (Given the value 

of the Force Field, Alphon Count of the Nucleus and the distance of the 
satellite from the nucleus, the student should be able to predict the 
distance at the next time) should not only focus the student's attention 
on the relevant information but should also affect the way this informa- 
tion is organized or processed so that it can be used in a new or transfer 
situation to make predictions. However, the mere statement of a rule: 

(The decrease in distance between each time is equal to the product of 
the Force Field and Alphon Count of the Nucleus) does not give the student 
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a clue as to how the iMiterial will be used or should be organized to 
facilitate its use. Thus, properly stated objectives may affect the 
student *s information processing and give him a transfer and/or reten- 
tion set. 

Based on the above argument, it was hypothesized that the 
presentation of objectives to the student wwjld first dispose the student 
to attend to the material related to the objective, and therefore reduce 
the total time and number of examples required to meet the objective, 
and second, give the student a transfer set which would enable him to 
score higher on transfer retrieval criterion measures. 

These effects, however ^ would be tempered or interact with 
the other properties of a given task. If objectives are inserted in a 
task which otherwise has minimal orienting or organizing stimuli, then 
the above hypothesized effects should be very evident. On the other 
hand, if objectives are inserted in a task which has other effective 
orienting stimuli, such as rules or pre-questions, then the objectives 
would be somewhat redundant and have a more subtle effect. However, 
the organizing effect of objectives should be evident even when other 
orienting stimuli are available. 

Relationship of Cognitive Abilities to Task Performance 

Recently many investigators have found a study of the rela- 
tionship between cognitive abilities and learning performance to be very 
helpful in understanding the effect of task variad)les on the learning 
process. Ferguson (1954 j 1956) was one of the first to propose a 
rationale behind this approach. He assumed that abilities were patterns 
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of behavior which had become relatively invariant because of overlearning. 
He further suggested that abilities may transfer differentially and exert 
their effects on learning tasks differentially. This approach received 
further emphasis when Cronbach (1957) recommended that investigators 
narrow the gup between correlational and experimental psychology and study 
the interaction between abilities and treatments. Bunderson (1967) con- 
ducted a study to investigate the relationship between abilities and per- 
formance in a concept learning task at different stages of practice. By 
analysing the task, Bunderson developed a conceptual model involving 
information-processing constructs euid postulated three higher-order pro- 
cesses: a problem analysis process, a search process, and an organization 
process. Thirty mental tests were chosen for their relevance to the model 
and were; administered to < the ^s. . The test scores were factor analyzed, 
and ten factors were interpreted. The performance scores at different 
stages of practice on the concept— attainment tasks were located within 
the factor structure by a factor extension procedure. The results 
revealed that the abilities did transfer at different stages of practice, 
giving support to the assiamption that the abilities related to those pro- 
cesses required at a given stage would transfer at that stage of practice. 

Dunham and Bunderson (1969) carried the approach one step 
further to discover if the relationship between cognitive abilities and 
performance in a concept learning task could be alterod by manipulating a 
task variable. They argued that if measures of a cognitive ability tap 
an underlying intellectual process then the rele.tionship between the 
ability and task performance must be due to an aspect of the task which 
requires that ability. Thus, if a task variable is manipulated, the 
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relationship between the ability *ind performance on the task may be 
changed. To test for a shift in ability relationships one group of Ss 
was given the decision rules for solving the concept problems, while a 
second group was not given the decision rules. Performance scores on the 
concept problems were extended into the factor space of six cognitive 
abilities considered to be relevant to the task. The results showed that 
the two conditions reqxiired the use of different abilities. A discriminant 
analysis of solvers from non-eolvers re- ».aled that Ss with one ability 
succeeded under one condition, while Ss with a different ability succeeded 
under the other condition. Thus, it was argued that "...the manipulation 
of the instructional variable resulted in a change in the nature of the 

information processing which occurred in the two groups." 

In an attempt to bridge the gap between the laboratory and the 

classroom Bunderson, Olivier, and Merrill (1970b) used an imaginary 
science called the Science of Xenograde as a hierarchical learning task 
to study the relationship of cognitive abilities to the manipulation of 
a task variable. Before learning the task, all £s were given a battery 
of cognitive tests. The Ss were then randomly assigned to two groups. 
Group I received a rule plus example instructional treatment while 
Group II received an example -only instructional treatment . All Ss 
received additional examf>les of a rule until they were able tc pass 
constructed response test items on the rule. The battery of cognitive 
tests were factor analyzed and -egression analysis of the factor scores 
and the criterion, number of examples, were conducted. Thf; results 
showed that memory and reasoning were related to the number of examples 
required by the example-only group but not related to the number of 
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examples required by the rule-example group. Thus, by manipulating a 
task variable it was possible to vary the nature of the learning process 
such that the constraints of memory and reasoning abilities were minimised. 
Studies, such_as those cited above, which investigate the 
relationship of abilities to task performance have other important impli- 
cations besides helping to understand the effect of task variables on 
the learning process. The relationships between abilities and perfor- 
mance also have important implications to the design of instruction. 

Snow (1969) argued that if we seek to individualize instruction so that 
each individual's performance is maximized on a given set of criteria, 
then we must search for evidence that it is worthwhile to instruct 
students differentially and discover those variables which will allow us 
to make classification decisions that will lead to improved instructional 

outcomes . 

Based on the results of the studies cited above it was hypo- 
thesized that a manipulation of the presentation of objectives to the 
students would exert an effect on the relationship between cognitive 
abilities and task performance. Analysis of such ability by treatment 
interactions would further reveal the actual effect of objectives on 

the learning process . 

An analysis of the information processing required to learn 
the task used in this study suggested that the following hypothesized 
processes would be required by Ss who receive only examples : 

1. Formation of a hypothesis as-, to what parts of an example 

are relevant; 

2. Inference of a rule which determines the relevant relation 




ships in the example; 
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3. Application of the inferred rule to predict entries in a 
new example; 

4. Recall past instances or examples of a rule; and 

5. Conjuntive utilization of several inferred rules to make 

correct predictions. 

On the basis of the above analysis it was hypothesized that the 
abilities or processes of induction, associative memory, and general 
reasoning would be highly related to task performance for Ss in an example- 

only treatment group. 

It was hypothesized that the presentation of objectives in 
addition to examples would allow Ss to focus on the relevant parts of 
an example, thus facilitating the inference process and reducing its 
relationship to task performance. This focusing effect would also reduce 
the importance of associative memory since fewer parts of an example would 
need to be recalled when studying future examples of the same ^e. The 
organizing effect of objectives would also reduce the requirement for 

general reasoning. 

If rules plus examples are presented to the £s, it was hypo 
thesized that the role of induction and associative memory would be 
reduced since the rule would eliminate the need to infer a rule or recall 
past examples. The role of general reasoning would also be reduced because 
of the additional structure provided by the precise statement of the rule 

governing the relationships demonstrated by the example. 

If both objectives and rules are presented to Ss, it was 
expected that the relationships between task performance and the abilities 
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of induction, associative memory, and general reasoning would be reduced 
as when only objectives or rules are included in the task. There may 
even be an additive reduction in the role of these abilities, although 
such an effect is not expected to be significant . 

Role of Objectives in the Learning by Discovery Issue 

In a very well known article Bruner (1961) hypothesized that 
learning by discovery allowed the student to organize those things which 
he learned in such a way that memory processing and transfer retrieval 
would be facilitated. The research which has been done (Wittrock, 1966) 
in an attempt to prove or disprove this hypothesis has produced con- 
flicting results. Most of this research has suffered from problems in 
design and the assumptions that all students learn best by one method 
and that the said method is best no matter what task is to be learned. 
Ausubel (1964) argued that these asstumptions are not plausible, and that 
few students are sufficiently brilliant to discover every thing they need 
to know. He further claimed that the miracle of culture is made possible 
only because it is so much less time consuming to communicate and explain 
an idea meaningfully to others than to require them to rediscover it by 
themselves . 

In addition to the results cited in the previous section, 
Bunderson et al., (1970) found that when both groups (rule-example vs. 
example-only) learned the hierarchical task equally well, the example- 
only group took significantly more examples to reach criterion than the 
rule-example group, and that there was no significant difference between 
the groups performance on either a retention test or a transfer test. 
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Therefore, even with more examples, the example-only group did not show 
superior performance. However a significant interaction (p ' .01) of 
reasoning factor scores with rule-example vs. example-only treatments was 
found using number of examples as criterion. These results indicate that 
some students with high reasoning ability performed better under the 
example-only treatment than under the rule -example treatment. In as much 
as the regression lines crossed at the high end of the range of reasoning 
abilities, the contention by Ausubel (1964), that we should not attempt 
to structure the learning environment of the non-exceptional child in 
terms of the educational objectives and teaching methods that ar« 

appropriate for a few, is supported. 

Even though the study cited above suggests that, for most 

students, learning by discovery not only does not produce superior per- 
formance on retention and transfer, but is also more time consuming than 
rule instruction, the question remains whether these results would be 
changed by the presentation of objectives to the student. Most educators 
would agree in principle with Ausubel (1964) that before students can 
discover concepts and generalizations with reasonable efficiency, they 
must be given problems which are structured and organized in such a way as 
to make discovery possible. Few students would be able to make sense out 
of masses of raw data. Based on the argument that objectives have a 
focusing and organizing effect, it was hypothesized that if specific 
objectives were given to a student learning by discovery, they would 
reduce his search time and allow him to organize relevant information in 
such a way as to enable him to have greater transfer retrieval. As argued 
in the previous section, the presentation of objectives would also reduce 
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the strain which learning by discovery places on the processes of 
induction and general reasoning. 

Statement of Hypotheses 

In the above sections several hypotheses were stated concerning 
the possible effects the presentation of objectives would have on the 
learning process. The following is a summary of the hypotheses made in 

this study. 

1 1 , No differences were expected between groups on the posttest 

since all Ss received additional examples until a minimum 
criterion performance was reached. 

2. It was expected that ihe significant reduction in the number 
of examples required by a rule group as compared to an 
example-only group in earlier studies would be replicated. 

3. In as much as the total time required to complete the task 
was expected to have a high positive correlation with the 
number of examples required to finish the task, it was further 
hypothesized that those groups who received statements of the 
rules would require significantly less time to learn the task 
than those groups who did not receive rules. 

4. It was hypothesized that the presentation of objectives would 
significantly reduce the number of examples and the amount 
of time required to complete the task. It was hypothesized 
that this reduction in time and examples would be greater 
when objectives were added to a task with no other orienting 
stimuli than it would be when objectives were added to a task 
with other orienting stimuli such as rules. 
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5. It was expected that the non- significant differences between 

a rule group and an example-only group on transfer test scores 
found in earlier studies would be replicated. 

6. It was hypothesized that a treatment group which received 
objectives in addition to examples would score significantly 
higher on both retention and transfer tests than treatments 
groups which received only examples or rules plus examples. 

7. It was hypothesized that the presentation of objectives and/or 
rules in addition to examples would significantly reduce .he 
relationship between task performance and scores on memory, 
induction, and general reasoning cognitive ability tests. 
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METHOD 



Subjects 

The 160 Ss who participated in this study were taken from four 
sections of an introductory educational psychology course and three sections 
of a science education course at The University of Texas at Austin. All 
Ss were required to participate as a class assignment. Only 131 of the 
original £s completed all three phases of the study; however, the data 
reported in this paper are based on only 130 Ss. During the original data 
analyses, it became apparent that the data for one S were highly discrepant 
from those of all other Ss. Her scores on the ability measures were con- 
sistently low, and she required 14 more examples and twice as much time to 
complete the task as any other S. Her scores on the post, retention, and 
transfer tests were also very low. Because of the highly discrepant nature 
of this S's data, they were excluded from the final data analyses. Without 
the exclusion of this outlier, the results would have been spurious. 

Ability Measures 

The studies reviewed in a previous section (Bunderson, 1967; 
Bunderson 6 Dunham, 1969; Bunderson et al., 1970b) were conducted to 
investigate the relationships of cognitive abilities to task performance. 

In these studies the learning tasks were analyzed to establish what cog- 
nitive processes were required to perform the tasks, and existing published 
tests were analyzed in order to select those which supposedly measured 
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these cognitive processes. However, Bunderson (1969b) has <jrgued that 
it may be more profitable to define and develop new measures which are 
more task relevant to assess the actual processes required by a given 
task. In this study, an effort was made to investigate further the 
validity of Bunderson *s argument by comparing the predictive power of 
three new task relevant tests developed by this author to that of three 
additional ones selected from existing published tests. The three pub- 
lished tests were selected from the Kit of Reference Tests for Cognitive 
Factors (French, Ekstrom, 6 Price, 1963). The First and Last Names Test 
was selected to measure associative memory; the Letter Sets Test was 
selected to measure induction; and the Ship Destination Test was selected 
to measure general reasoning. 

As mentioned above, three new tests were developed for this 
study in order to assess directly the actual processes required by the 
task. Each of these new tests required the £s to process the same type 
of information as must be processed in the learning task, while the pub- 
lished tests required similar processes on information not related to 
the task. The First and Last Names test required the memorization of 
names, while the learning task and the new memory test (Memory of Number 
Series Test) both required the memorization of number series. The learn- 
ing task and the new induction test (Bi-Column Number Series Test) both 
required the inference of rules from related columns of numbers while 
the Letter Sets Test required the inference of rules based on sets of 
letter combinations. The new general reasoning test (Tote Mobile Test) 
required the structuring and application of rules which were isomorphic 
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to those found in che learning task. The instructions and sample items 
from th? new task relevant tests may be found in Appendix G. 

Experimental Task and Materials 

The hierarchical learning task used in this study was an imaginary 
science called the Science of Xenograde Systems. The use of an imaginary 
science assured that none of the S^s had any previous experience with the 
task and eliminated the necf.ssity of pretesting and discarding Ss due to 
prior knowledge of the task. Since the structure and content of the 
Science was similar to that of formal science topics, the generality of 
the results was increased. The initial version of the Science was devel- 
oped by Carl Bereiter at the Training Research Laboratory, University of 
Illinois, for use in studying group interaction problems. This skeleton 
version was further expanded and developed by David Merrill (1964). 

Merrill's version of the Science was simplified, and an instructional pro- 
gram for presenting the task on the IBM 1500/1800 Computer-Assisted 
Instruction System was designed by this author and William Olivier accord- 
ing to an instructional design model developed by Bunderson (1969). 

In the current version of the Science, a Xenograde System consists 
of a nucleus with an orbiting satellite. The satellite is composed of 
small particles called alphons which may also reside in the nucleus. Under 
certain conditions, a satellite may collide with the nucleus. When such 
a collision occurs, a "blip" is said to have occurred, and the satellite 
may exchange alphons with the nucleus. The subject matter of the Science 
deals with the principles or rules by which the activity of satellites 
and alphons may be predicted. 
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The terminal objective of the task required that predict and 
record the state of the alphons and satellite of a Xenograde System at 
successive time intervals given the initial state of the System at time 
zero. The sub-objectives and hierarchical sequence for presenting the 
principles or rules of the Science were determined by en analysis of an 
efficient information processing algorithm for performing the terminal 
behavior. This analysis entailed the determination of a series of ordered 
steps comprising an algorithm which S would use to perform the terminal 
behavior. The next step of the analysis entailed the specification of 
the exact information which would be required by S in order tc make a 
correct response at each succeeding step of the algorithm. The sub- 
objectives of the task were then developed to correspond with the steps 
of the algorithm, and the sequence for presenting the principles or rules 
(information) required to perform the sub-behaviors was made to corre- 
spond with the order of the steps . 

The instructional program consisted of ten modules corresponding 

to the ten steps of the algorithm. The materials for each module included 
a statement of a sub-objective, a statement of a rule, five examples of 
the rule, and five short constructed response tests. The examples were 
in the form of partial Xenograde tables x^?hich showed the activity and 
relationships of a Xenograde System at several points in time. The latter 
part of Appendix B contains a sample Xenograde table with an explanation 
of how it is to be interpreted. A sample test can be found in Appendix C, 
and a statement of the sub-objectives and rules of the Science are found 
in Appendix A. A printed instruction booklet was also provided which 
contained an introduction to the Science, the purpose and justification 
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of the course, instructions on reading Xenograde tables, and a treatment- 
specific explanation of the procedure for learning the task. A sample 
booklet is found in Appendix B. The method used for presenting these 
materials is discussed in the procedure section. A more ccxnplete des- 
cription of the task and copies of the examples ai.i test items may be 
found in Bunderson, Olivier, and Merrill (1970a). 

Dependent Measures 

Posttest— Retention Test . In previous studies (Bunderson 
et al., 1970b; Olivier, 1970) using this task, the posttest was designed 
to provide corrective feedback to the Ss while taking the test. This was 
done to prevent cumulative errors. Since Olivier (1970) found that this 
feedback had an instructional effect, a new posttest and retention test 
were developed. The posttest and retention test in this study were 
parallel forms with constructed response test items which required £s 
to predict the successive state of the alphons and satellite of a Xeno- 
grade System by making entries in a Xenograde Table. These entries could 
be determined by using the rules of the Science and the given previous 
entries of the table. The items were sequenced so as to simulate the 
processing of a continuous algorithm, but each item was independent to 
avoid the necessity of providing corrective feedback. Appendix D contains 
examples of instructions and items for the post and retention tests. 

Transfer Test . The transfer task consisted of a booklet con- 
taining two Xenograde Tables and 24 constructed response test items. The 
Ss were required to infer three higher-order rules of the Science from 
the tables and made predictions based on the inferred rules. The transfer 
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test score was the total number of correct predictions made by S . 
instructions and sample items for the transfer test are found in 

Appendix E. 

Equipment 

The instructional program Jas written in the Coursewriter II 
language and presented to the Ss by the IBM 1500/1800 Computer-Assisted 
instruction System. The rules and objectives of the Science were displayed 
on the ir.'l 1512 random access image projector, while the exanr>les of the 
rules were displayed on the cathode xay tube of the IBM 1510 instructional 
terminal. The use of a computer-assisted instruction system to present 
the instructional material made it possible to run up to seven Ss at 
one time while maintaining tight control over the variable stimulus events 
for each S and simultaneously recording accurate latencies and responses. 



Proced\3re 

The six ability tests were administered to all Ss in several 
group sessions. Immediately preceding the testing, a short lecture was 
given to the Ss to explain the value of their participation in the study 
and give an introduction to computer-assisted instruction. The Ss were 
randomly assigned to four groups: an Example-Only group (n=32), an 

Objective-Example group (n=33), a Rule-Example group (n-32), and an 
Objective-Rule-Exaiq>le group (n=33). Fig. 1 is a graphical representa i 
of the 2 X 2 factorial design formed by these groups. Following the 
testing session all Ss made individual appointments for two sessions, 
separated by two weeks, at the Computer-Assisted Instruction Laboratory. 
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Figure 1. — 2 x 2 Factorial design used in this study. 
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During the first session, all Ss first received a printed instruction 
booklet corresponding to their group assignment and were instructed and 
tested on the use of the terminal. Next, the ^s learned the Science 
and took the posttest. 

In learning the Science, ^s in the Example-Only group received 
an example of the first rule of the Science displayed on a cathode ray 
tube. After studying the example each ^ responded to a three item con- 
structed response test where he was required to predict certain values 
using the rule inferred from the example . If the ^ responded correctly 
to two out of the three test items he was given an example of the next 
rule in the sequence; otherwise, he was given another example of the same 
rule followed by another three item test. This sequence of new examples 
followed by a test continued until the ^ responded correctly to two out 
of the three items or received five examples. The task was completed 
after all 10 rules of the Science were learned to the required criterion. 

The Ss in the other three groups learned the science by the 
same basic procedure except for the following treatment differences: 

The Objective-Example group was shown a statement of a sub-objective on 
the image projector while the corresponding example was displayed on the 
cathode ray t\Jbe; the Rule-Example group was displayed a statement of the 
rule corresponding to each example; and the 0 b 3 ective-Rule-Example group 
received both the objective and the rule in addition to the example. 

Two weeks after their first session, all ^s returned to the 
laboratory and took the retention and transfer tests. 



RESULTS 



Descriptive Statistics 

In addition to total scores on the six cognitive ability tests, 
posttest, retention test, and transfer test mentioned in the previous sec- 
tion, data were obtained for each ^ on the following criteria: total number 

of examples required to learn the Science, display latency, test-item- 
response latency, and total latency. Display latency was the total time 
S spent studying the examples , and depending upon s treatment group , 
the corresponding rules and/or objectives. Test-item-response latency 
was the total time required by £ to respond to the three- item tests fol- 
lowing each example display. Total latency was merely the sum of the 
display and test-item-response latencies. Display and total latency data 
for three Ss were lost. Therefore, all analyses cn these criteria are 
based on a n = 30 for the Example-Only group and a n = 31 for the 
Objective-Example group. An inter correlation matrix of all criterion 

variables is found in Table 23 of Appendix H. 

The descriptive statistics and reliabilities of the ability tests 

are found in Table 1. Time constraints made it impossible to administer 
parallel forms of these tests. The reliabilities were therefore estimated 
using the Kuder- Richard son formula 20 (KR-20). These were not pure speeded 
tests, but they were timed, and all Ss were not able to attempt all items 
in the time allowed. Since the reliabilities estimated by KR-20 may in 
some cases be higher than would have been obtained from alternative forms, 
the communality is reported as a lower-bound estimated of the reliability. 
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The reliabilities of the posttest, retention test, and transfer 
test are presented in Table 2. These reliabilities were also estimated 
using the Kuder Richardson formula 20; however, none of the criterion tests 
were timed. The correlation between the posttest and retention test, which 
were parallel forms administered two weeks apart, was .82, 

The battery of cognitive ability tests were factor analyzed, but 
consistent with previous findings (Bunderson et al. , 1970b), it was not 
possible to separate out the three factors of associative memory, induction, 
and general reasoning. Therefore, a two-factor varimax solution which 
yielded the factors of reasoning and associative memory is presented in 
Table 3. The reasoning factor is marked by the two induction and the two 
general reasoning tests. An intercorrelation matrix of the six ability 
tests is found in Table 4. The correlations between the ability covariables 
and the criterion measures within each treatment groups may be found in 
Table 24 of Appendix H. 

Table 5 contains, by treatment group, the squared multiple corre- 
lations between three different subsets of the battery of cognitive abilities 
and each of the seven criterion measures used in the present study. The 
first row of each part contains the squared multiple correlations between 
the full battery of tests and each criterion based on the Ss in each respec- 
tive treatment group. The second row of each part contains the squared 
multiple correlations between the three published tests and each criterion 
while the third row of each part contains the squared multiple correlations 
between the three new task relevant tests developed for this study and each 
criterion. In general, the subset containing the task relevant tests 
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correlates higher on most of the criteria than does the subset of published 
tests for all treatment groups except the Rule-Example group. For the 
Rule-Example group, the correlations between the criteria and the published 
tests are generally higher than those between the criteria and the task 

relevant tests. 
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Table 1 

Descriptive Statistics of Ability Measures 



Tests 


Number of 
Items 


Means 


SD 


Reliability 

(KR-20) 


Cofiimunality 


Memory of Number Series 


15 


6.8 


3.0 


.69 


.73 


(MA) 

First and Last Names 


15 


11.6 


3.4 


.83 


.72 


(MA) 

Bi -Column Number Series 


15 


4.1 


1.7 


.53 


.37 


(I) 

Letter Sets (I) 


15 


10.4 


2.1 


.57 


.53 


Tote Mobile (R) 


15 


6.4 


2.2 


.73 


.58 


Ship Destination (R) 


24 


13.6 


4.1 


.85 


.68 



Table 2 

Reliabilities of Post, Retention, and Transfer Tests 





Post 


Retention 


Transfer 


Reliability 


.92 


.93 


.80 


(KR-20) 



Table 3 

Varimax Rotation Factor Matrix® 





Factor Lc 


ladings 


Tests 


Reasoning Factor 


Associative Memory Factor 


Memory of Number Series 


1877 


8336 


First and Last Names Test 


0078 


8465 


Bi -Column Number Series 


6001 


0802 


Letter Sets 


7006 


1954 


Tote Mobile 


7458 


lb07 


Ship Destination 


8191 


-1103 




^Decimal points are omitted. 
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Table 4 

Intercorrelations of Six Cognitive Ability 



Tests 


1 2 


% 


4 


s 


6 


1. 


Memory Number Series (MA) 


1.00 45** 


12 


31** 


23t* 


07 


2. 


First and Last Names (MA) 


1.00 


12 


07 


14* 


02 


3. 


Bi-Colwn Number Scries (I) 




L.OO 


2B** 


28** 


34** 


4. 


Letter Sets (I) 






1.00 


39®* 


41** 


5. 


Tote Mobile (R) 








1.00 


49** 


6. 


Ship Destination (R) 










l.OO 



^Decimal points are omitted. 

= 208. The additional T8 Ss tested were from the same population, but 
participated in another study. 

< .01 



Table 5 

Squared Multiple Correlations Between Subsets of Ability Test Battery and Criterion Measures' 



Test Subset 


Posttest 


Retention 

Test 


Transfer 

Test 


Number of 
Examples 


Display 

Latency 


Test 

Latency 


Total 

Latency 






Example-Only C2i» 










All 


27 


48 


36 


42 


35 


51 


47 


Published 


21 


16 


22 


39 


24 


30 


27 


Task Relevant 


23 


44 


29 


26 


29 


46 


40 






Objective-Example Group 








All 


21 


20 


24 


27 


25 


21 


22 


Published 


07 


06 


16 


20 


20 


11 


12 


Task Relevant 


21 


18 


16 


16 


24 


10 


17 






Rule-Example Group 








All 


24 


17 


31 


35 


51 


45 


52 


Published 


24 


16 


30 


18 


40 


38 


42 


Taak Relevant 


11 


06 


13 


29 


33 


22 


30 



Objectivc-Rule-ExaBDple Group 



All 


31 


39 


36 


51 


47 


41 


so 


Published 


15 


29 


16 


14 


32 


SO 


36 


Task Relevant 


25 


24 


20 


39 


41 


36 


44 



dn 



^Decimal points omitted. 






Analysis of Ability by Treatment Interactions 

In order to investigate the hypothesized ability by treatment 
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interactions, the relationships between the abilities and task performance 
was operationalized in terms of the slope (amount of change in the criterion 
per unit change in the covariable) of the regression lines for each of the 
treatment groups. Thus, the hypothesis that the relationship between per- 
formance and a given ability would be reduced by the availability of 
objectives and/or rules was .accepted if the slopes of the regression 
lines for the objective and/or rule groups were significantly less than 
the slope of the regression line for the Example-Only group. Linear 
regression analysis (Bottenberg 6 Ward, 1963) was used to test for dif- 
ferences in the slopes of treatment group regression lines. 

The following series of comparisons were made using the reasoning 
and memory factors plus the six individual ability tests as covariables 



with each of the seven criteria; 

1. The error sums of squares of the residual vector of a full 
model which allowed the slopes for all regression lines to 
be different (Model 1) was compared with the corresponding 
error sums of squares of a restricted model (Model 2) which 
assumed equality of slopes for all regression lines. The 
resulting F statistic is labeled in the regression analyses 

tables as Fi . 

2. If Fj was significant, then the ability by treatment inter- 
action was further analyzed by comparing the error sums of 
squares of the full model (Model 1) with the error sums of 



41 






28 

the squares of a second restricted model (Model 3). Model 3 
assumed that the regression lines for the Objective-Example 
group, the Rule-Example group, and the Objective-Rule- 
Example group were all mutually parallel but allowed the 
slope of the regression line for the Example-Only group to 
be different. The resulting F statistic is labeled F2. 

3. If F2 was non- significant , then Model 3 was concluded to be 
true and was used as a full model to compare with the re- 
stricted model (Model 2) which assumed equality of slopes 
for all regression lines. If the resulting F^ statistic, 

F3 was significant, then it vras concluded that the slope 
for the Example-Only group was significantly different 
from the slopes of the regression lines for the other three 
treatment groups, 

The models described above are defined mathematically in 
Appendix G. The results of these analyses will be reported in the appro- 
priate sections which follow. 

Treatment Effects on Posttest and Retention Test Scores 

Since the experimental procedure required all to perform at a 
minimum criterion level on each rule before proceeding to the next, no 
group mean differences were expected on the posttest. This expectation 
was confirmed by a non-significant F from a random groups analysis of vari 
ance. Table 6 contains the group means and standard deviations for the 
posttest . 
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There also were no significant differences between the treatment 
groups on retention test scores. The retention test group means and standard 
deviations are given in Table 7 . 

No significant ability by treatment interactions were found using 
either posttest sco: j:s or retention test scores as criterion and individual 
ability test scores or factor scores as covariaoles • 

Treatment Effects on Transfei- 

Table 8 presents means and standard deviations “"or the transfer 
test. These data were evaluated by a 2 x 2 factorial analysis of variance 
which is summarized in Table 9. It should be noted that the significant 
rule effect is in favor of the groups which received a statement of the 
rules and not in favor of the no-rule groups as would be expected by advo- 
cates of the discovery hypothesis. Tht, objective effect did not reach 
significance at an acceptable level, but it did approach significance, 

F(l, 126) = 3.1, 2 . ^ with the objective groups obtaining higher mean 

transfer scores than the no-objective groups. There were no significant 
ability by treatment interactions using the transfer test scores as 
criterion. 
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FosTlest Jroup Heans and Star^Sard Deviations 





Pules 


0! '.ect ives 


Nc 


Ye 






Moans 


..D 


Means 


sr 


No 


•♦5.4 


4.9 


45.1 


12.9 


Ye> 


44 . 1 


U.7 


47.8 


12.: 



Table 7 

RpTention Te-t .'.roup and Standard Deviations 



01 ^ectives 


Rules 


No 


Yc 


IS 


Means 


LL 


Means 


‘* *D 


N. 

Y.n 


44.2 

43.3 

j 


7.2 

13.^^ 


43. e 
46.2 


14.2 

i:.2 



ril 



Tr T. * 'l.-ans and ‘rand^ird D^viatioi;- 



Mu le 







i le t i ve^ 


Nw 


Yes 




Moan^ 


r 








1 1 . 




14.1 






iJ .2 

. 




14.7 





*'aJ le 



of VarMT'Ct* irv for Trind.-r t .ere 



, ourc-' 



df *1S f 



Eo^ w»--en 

:* t i vt- . ( A) 
?uU- (B) 

A / : 

• . r h ; r. 

T:)*al 



3 81. »’ 

1 . 7 .^ 

1 17C.2 

1 1 

126 21.9 

:3.- 
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Treatinent Effects on Nuniber of Examples 

The treatment effects on the number of examples required to 
learn the task is graphically portrayed by the group frequency distri- 
butions given in Fig. 2. The corresponding means and standard devia- 
tions are presented in Table 10, while the 2 x 2 analysis of variance 
results are reported in Table 11. The significant rule effect repli- 
cates the findings of earlier studies (Bundorson et 1970b) which 

revealed that the presentation of rules significantly reduces the num- 
ber of examples required to learn the task. The significant objective 
effect shows that the presentation of objectives also reduces the num- 
ber of examples required, but this reduction is noc nearly as marked 
as the reduction caused by the pr'^isentation of the rules. 
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NUMBER or EXAMPLES 

Figure 2. --Frequency Distribution by Treatment Group of the Number of Examples Received. 



Table 10 

Group Means and Standard Deviations of the Number of 
Examples Required to Learn the Task 






Rules 


Objectives 


No 


Yes 




Means 


SD 


Means 


LO 


Ho 


15.0 


3.6 


11.0 


2.D 


Yes 


13.3 


3.4 


10.6 


1.2 



Table 11 

Analysis of Variance Summary for Number of Examples 



Source 


df 


MS 


F 


Between 


3 


137.9 




Objectives (A) 


1 


35.9 


4 . /*'• 


Rules (3) 


X 


364.3 


48. 


A X B 


1 


13.8 


1.8 


Within 


126 


7.5 




Total 


129 


10.5 





< .05 
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Although there were no ability by treatment interactions which 
reached the .05 level of significance using number of examples as criterion, 
the regression analysis using reasoning and memory factor scores as covari- 
ables are reported in order to give the reader a better feel for the actual 
relationships between the abilities and treatments. A summary of the 
regression analysis of reasoning factor scores as the covariable and num- 
ber of examples as the criterion is presented in Table 12. (Since 
was not significant, F 2 and F 3 are not reported.) The equations for the 
treatment group regression lines, the criterion and covariable group 
means, and the ranges of scores on the criterion and the covariable are 
also reported in Table 12. The corresponding regression lines are plotted 
in Fig. 3. The results of the regression analysis using memory factor 
scores as the covariable are presented in Table 13, while the regression 
lines are plotted in Fig. 4. 

From Fig. 4, it should be noted that the slopes of the regres- 
sion lines for all treatments were very close to zero, which indicates 
that there was very little relationship between S’s task performance and 
his memory factor scores. In contrast, the interaction of reasoning 
factor scores with the instructional treatments (Fig. 3) approaches sig- 
nificance, F(3, 122) = 2.18, £ < .10, with the slopes of the Example-Only 
and Objective group regression lines being somewhat steeper than those of 
the Rule and Obj ective-Rule -groups . 
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REASONING FACTOR SCORES 

Figure 3. -Interaction of Reasoning Factor Scores and Treatments with Number 
of Examples as Criterion. 



Table 12 

Regression Analysis Summary for Number of Examples with Reasoning Factor Scores as Covariable 



Group 


Means 


Range 


Equations for Group 
Regression Lines 


Criterion 


Covariable 


Criterion 


Covariable 


Example-Only 


15.00 


-.215 


10 to 24 


-2.9 to 2.2 


X = -1.48A + 14.69 


Ob j e ct ive -Examp le 


13.30 


.046 


10 to 24 


“2.0 to 1.6 


0 = -1.51A + 13.37 


Rule-Example 


11.00 


-.146 


10 to 20 


-3.1 to 1.9’ 


R = -0.75A + 10.89 


Ob j ect ive -Rule-Examp le 


10.61 


.257 


10 to 14 


-1.9 to 2.2 


RO = -0.20A + 10.66 






Comparison^ 


dfi 


df2 


RSQf 


RSQit 


F 



3 122 .4434 .4136 2.18 



^Comparisons are described under section titled Analysis of Abilit y by Treatment Interactio n s . 
The mathematical definitions of the linear regression models are round in Appendix G. 
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memory factor scores 

F' -ri '+. Interaction of Memory Factor Scores and Treatments with Number 

of Examples as Criterion. 



Table 13 

Regression Analysis Summary for Number of Examples with Memory Factor Scores as Covariable 



Group 


Means 


Range 


Equations for Group 


Criterion 


Covariable 


Criterion 


Covariable 


Regression Lines 


Example-Only 


15.00 


.057 


10 to 24 


-1.9 to 1.6 


X = -.94A + 15.15 


Objective-Example 


13.30 


.109 


10 to 24 


-2.2 to 1.9 


0 = -.40A + 13.44 


Rule-Example 


11.00 


-.242 


10 to 20 


-2.2 to 2.1 


R = -.40A + 10.90 


Objective-Rule-Example 


10.61 


-.131 


10 to 14 


-2.0 to 2.0 


RO = -.24A + 10.57 






Comparison^ 


dfi 


dfa 


FSQp 


RSQr 


F 


Tl 


3 


122 


.3331 


.3259 


• 44 



“Comparisons are described under section titl'd Analysis of AbiJi^ by T reatment Interactions. 
The mathematical definitions of the linear regressio.) models are found in Appendix G. 
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Treatment Effects on Latency 

Table 14 contains the means and standard deviations for each 
group on the three latency measures while Table 15, 16, and 17 give the 
corresponding results of the analyses of variance of the data using 
2 X 2 factorial designs. There was a significant rule effect on all 
three measures with the rule groups taking considerably less time to 
study the displays and respond to the criterion test items. The objec- 
tive effect is significant only on test-item-response latency. The 
groups which received objectives required less time to respond to the 
test items than the groups which were not presented the objectives. 

There also was a significant interaction (Table 14) with test-response 
latency as the criterion. This interaction indicates that the objec- 
tives have a greater effect in reducing response latency when added to 
a task which has no other focusing or organizing stimuli than they do 
when added to a task which has other effective orienting stimuli such 
as rules. In other words, the difference in response latency between 
the Example-only and Objective groups is greater than the corresponding 
difference between the Rule and Objective-Rule groups. 

In contrast to the objective effect on test-item-response 
latency where objectives reduced response time, the effect of present- 
ing objectives was in the opposite direction on display latency. Even 
though this effect did not reach an acceptable level of slgnlrlcance , 
the contrast between the order of the display latency means and the 
test-item-response latency means has theoretical significance which 
will be presented in the Discussion Section. In as much as objectives 
had opposite effects on the two component latencies, the combined total 
latency objective effect was non- significant (Table 17). 



Table 14 
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^iroup Means and Standard Deviations for Display UiTency» 
Test'Item-Response Latency, and Total Latenrv 





Latency 




r*-’up 


Display 


Tost 


Response 


Tot a 


1 1 




Means 


SD 


r 

Means 


SD 


Keans 


SD 


Example-Oni’, 


821.2 


37 9.8 


^23.3 


4:c.7 


j.^72.8 


771.7 


Ob ^ective-Lxamplo 


865.8 


377 S 


fj49.3 


2b3.3 


1-13." 




-ule- Example 


543 . 3 


208. h 


49 J. 7 


211.7 


1037. r 


3 '^.1 


1 u‘ot . ve-Rule*rxample 


b34.3 


218.2 


419.8 


128.2 


1053. 


321.2 


— — — — — 



le i 1' 

AnalV'^is of Variance Summary for Pisrlay Latency 



Source 


df 




MS 


F 


Between 


3 




7377 .8 




Objectives (A) 


1 




1445.3 


1.5 


Ruj.o'^ (B) 


1 




2 C 521. 5 


^ ^ ^ oft A A 


A X B 


1 




166.6 




Within 


123 




932.9 




Total 


126 




1086. 3 






Table 


16 






i 1 


- if ’’ir'^nce ‘-u^marv 


for 


Ter.t-T ten - * O'' 


ponse Lilt "nr*, 


"'Ource 


If 




MS 


r 


'etweep 


3 




16152.3 




O’' ' ^ct Ivc." ( A ) 






9365.2 




-ulei (?) 


1 




3536.1.8 


u 5 . 8*** 


A •. c 


- 




3'30.‘^ 


4.2* 


2’ ! 1 2 n 


120 




772 .2 




Total 


123 




1129.9 




*£ < .05 










A*Al < . Oul 


Table 
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Analysis of Variance 


Surnmary for Total 


Latency 


Source 


uf 




ME 


r 


Between 


3 




41315.2 




Ob ivOo (A) 


1 




4696.1 


1.6 


Rules (B/. 


1 




113245.9 


39.2*** 


A X B 


1 




6003.7 


2.1 


Within 


123 




2691.4 




Total 


12b 




3806.3 





< .JCl 
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Several significant ability by treatment interactions were 
found using latency as the criterion measure. Tables 18 through 21 report 
the results of the regression analyses using test-ittm-resporse latency 
as the criterion measure, while Figures 5 through 8 show the correspond- 
ing plots of the regression lines. 

Fig. 5 dramatically shows that test-item-response latency has 
a high negative relationship to reasoning, as defined by the reasoning 
factor scores, for £s in the Example-Only group. However, the regres- 
sion analysis (Table 18) shows that the corresponding relationship 
between reasoning factor scores and test-item-response latency is sig- 
nificantly reduced for Ss in the other three treatments . Similar 
results were obtained using the Tote Mobile Test scores (Table 19 and 
Fig. 6), the Ship Destination Test scores (Table 20 and Fig. 7), and 
the Letter Sets Test Scores (Table 21 and Fig. 8) as the covariables 

There were no significant interactions using display latency 
as the criterion measure; however, one significant interaction was obtained 
using total latency as criterion. The regression analysis results are 
reported in Table 22 with the regression lines plotted in Fig. 9. As can 
be seen from Fig. 9, the ability by treatment interaction found with LOtal 
latency as criterion is generally the same as that found using test-item- 
response latency as the criterion and Letter Sets Test scores as the co- 
variable (Table 21 and Fig. 8). In both cases, the slope of the Example- 
Only treatment group regression line was found to be significantly greater 
than the slopes of the regression lines for the other three treatment 
groups . 
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Figure 5 . --Interaction of Reasoning Factor Scores and Treatments with Test 
Item-Response Latency as Criterion. 



Table 18 



Regression Analysis Summary for Test-Itero-Response Latency® with Reasoning Factor Scenes as Covaria’ le 





Means 


Range 


Equations for Group 


Group 


Criterion 


Covariable 


Criterion 


Covariable 


Regression Lines 


Example-Only 


923.31 


-.215 


372 to 2025 


-2.9 to 2.2 


X = -203. 5A + 879.5 


Qbjective-Exanple 


649.34 


.046 


274 to 1224 


-2.0 to 1.6 


0 = - 80. 7A + 652.8 


RuXe*£xa^le 


493.70 


-.146 


285 to 1494 


-3.1 ro 1.9 


R = - 86. 3A + 481.2 


Objective-Ruie-Example 


419.17 


.257 


230 to 848 


-1.9 to 2.2 


RO = - 55. lA + 433.3 



Comparison^ 


dfi 


df* 


RSQr 


RSQr 


F 




3 


122 


.5078 


.4596 


3.16* 


2 


122 


.5078 


.5065 


.16 


T3 


1 


124 


.5065 


^.4696 


9.28** 


^Latency reported in 
^Compar isoiis are des( 


seconds . 


section titled Analysis 


of Ability by Treatment Interactions The 


mathematical definitions 


of the linear regression 


models are 


found in Appendix G< 
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Figure 6 . -interactions Between Tote Mobile Test Scores and Treatments 
with Test-Item-Pesponse Latency as Criterion. 



Table 19 



Regression Analysis 



Summary for Test-Item-Response Utency 



* with Tote Mobile Test Scores as Covariable 





Means 


Ran 




Equations for Group 

■e~>^ n SI 1 nVk 


Group 


Criterion 


Covari^dble 


Criterion 


Covariable 




Example-Only 


923.31 


6.09 


372 to 2025 


0 to 12 


X = -79, 5A + 1401.6 


Objective-Example 


649.34 


6.36 


274 to 1224 


3 to 10 


0 = ' 4.1A + 675.8 


Rule -Example 


493.70 


6.50 


285 to 1494 


2 to 11 


R = -32. 5A + 704.8 


Objective-Rule-Example 


419.17 


6.52 


230 to 848 


2 to 11 


RO = -24. 5A + 579.4 




^Latency reported in seconds. 

Wilsons d..crlb,d .nd«- s.«i=d «.».lvdU .( dblllty b, l.,.r.d,l..d . Th. 

of .1.. lln.» WSion ~i.l. found .n ,pp.nd.< C. 
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Figure 7. --Interaction of Ship Destination Test Scores and Treatments with 
Test-It«a-Response Latency as Criterion. 



Table 20 

Regression Analysis Summary for Test-Item-Response Latency^ with Ship Destination Test Scores as Covariable 



Group 


Means 


Range 


Equations for Group 
Regression Lines 


CrlterloQ 


Covariable 


D'lterlon 


Covarlable 


Lx«BpI«-Only 


923.31 


12.81 


372 to 2025 


2 to 19 


X = -49. 9A ♦ 1562.5 


^ j active -Example 


649.34 


13.24 


274 to 1224 


4 to 18 


0 = -19. 2A 903.0 


KUe-&(ai^Ie 


493.70 


13.56 


285 to 1494 


2 to 19 


R = -21. 2A + 781.4 


Objective-Rule-Example 


419.17 


15.33 


230 to 848 


7 to 20 


RO = - 7.3A + 531.8 



Comparisoo^ 


dfi 


dfa 


RSQp 


BSQr 


F 


fi 


3 


122 


.47 57 


.4396 


2.80* 


f: 


2 


122 


.4757 


.4724 


.38 




1 


124 


.47 24 


.4396 


7.71** 



^Latency reported in seconds.; 

^Comparisons are described under section tHled Analysis of Ability by Treatment In teractions.- 
The mathematical definitions of the linear regression models are found in Appendix G.: 



*£ < .05 

**£ < .01 
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Figure 8 . -Interactions Between Letter Sets Test Scores and Treatrnents 
with Test- Item-Response Latency as Criterion-: 



Table 21 



Regression Analysis Sumnary for Test- Item-Response Latency® with Letter Sets Test Scores as Covariable 



Group 


Iteasis 


tazica 


£<}uations for Group 
tofresaioo Lines 


Or i tor ion 


Covariable 


Criterion 


Covariable 


Ciuffplo-Only 


923.31 


10.25 


372 to 2025 


6 to 14 


X = --11.16A ♦ 2062.82 


Objoctiva-Exan^la 


649,34 


10.55 


274 to 1224 


6 to 14 


0 = - 10.29A + 756.70 


nuLU^Cxauplo 


493.70 


9.62 


285 to 1494 


0 to 15 


R s - 14.05A ♦ 628.96 


Ob joct ive-Rule-Exdnple 


419.17 


10.73 


230 to 848 


6 to 15 


RO = - 34.97A + 794.32 










dfi 


dfs 


KSQp 


RSQr 


F 




3 


122 


.4741 


.4035 


5.47** 


Fi 


2 


122 


.4741 


.4710 


.37 


Fj 


1 


124 


.4710 


.4035 


15.83*** 



^Latency reported "'n seconds. 

^Comparisons are describ-sd under section titl^ Analyr-is of Ability by Treawient Interactions . 
The mathematical definitions of the linear regression models are found in Appendix G.- 
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Figure 9.— Interaction of Letter Sets Test Scores and Treatments 
with Total Latency as Criterion. 



Table 22 

Regression Analysis Summary for Total Latency^ with Letter Sets Test Scores as Cc' ariable 



Group 



Cx«nplc*0nly 

Objective-Example 

ftule-Cxaeiple 

Ob j ect ive -Rule-Example 



Means 



Criteria 



1772.76 

1513.47 
1037.58 

1053.48 



Cover iable 



10.25 

10.55 

9.62 

10.73 



Range 



Criterion 



711 to 3442 
673 to 2859 
566 to 2687 
547 to 2097 



Covariable 



6 to 14 
6 to 14 
0 to 15 
6 to 15 



Equations for Group 
tegression Lines 



X = -183. 37A + 3637.28 
0 = - 11.71A ♦ 1636.66 
R = - 28.87A ♦ 1315.46 
RO = -106. 65A ♦ 2197.45 



Comparisoo^ 



dfi 



dft 









Fi 

F2 

Fs 



119 

119 

121 



.3936 

.3936 

.3800 



.3345 

.3800 

.3348 



3.85* 

1.33 

8.83** 



^Latency reported in seconds.: 

^Comparisons are described under section titled Anu ysls of Ability by Treatment Interact iqnj . 
The mathematical definitions of the linear regression models are found in Appendix 

*£ < .05 

**£ < .01 
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DISCUSSION 




In the introductory section of this paper it was proposed that 
objectives would serve as orienting and/or organizing stimuli which 
dispose Ss to attend to and organize relevant aspects of given information 
so as to facilitate attainment of the objectives. Therefore, it was 
hypothesized that the orienting effect of objectives would reduce the 
number of examples and amount of time required to complete the task, while 
the organizing effect of objectives would improve performance on transfer 
retrieval measures. It was also hypothesized that the orienting effect 
of objectives would compensate for low associative memory and induction 
abilities while the organizing effect of objectives would compensate for 
low reasoning ability Thus the relationship between these abilities, 
assumed to be required by the task, and task performance would be reduced. 

It was further expected that the effects of inserting objectives 
in a task without other orienting or organizing stimuli would be greater 
tnan the effects of inserting objectives in a task with other orienting 
stimuli such as rules. The effects of rules found in previous studies 
(Bunderson et al., 1970b) were expected to be replicated in this study, 
i.e., rules would reduce the number of examples required to complete the 

task and reduce the requirement for general reasoning. 

The design of the present study was such that all Ss were required 

to reach a minimum criterion performance at each level of the task before 
they were allowed to go on to the next level. This procedure was used to 
assure that all treatment groups would perform at the same level on the 
posttest. Unless all groups learned the original task equally well, 

44 
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transfer measures could not be 



differential performance on retention or 
interpreted in terms of the organization or structure provided by an 
instructional treatment. The results confirmed the expectation of 
non-significant group differences on posttest performance. Therefore, 
it was concluded that all groups had learned the task equally well. 

The hypothesis that the Objective-Example group would perform 
significantly higher on the retention and transfer tests than other 
treatment groups was not supported. Since there was a negligible 
decrement in performance between the posttest and retention test for 
all treatment groups, the retention interval of two weeks may have been 
too short for the treatments to have had an affect on retention. However, 
there were significant treatment effects on transfer performance. Even 
though the rtile groups receivea significantly fewer examples and took 
significantly less time to learn the task, their performance on the 
transfer test was significantly higher than that of the no-rule groups. 

If we compare the objective groups and the no-objective groups, the same 
type of result is obtained even though the differential performance on 

the transfer test only approaches significance . 

As mentioned in the previous section, the significant rule 
effect in favor of the rule groups on the transfer measure does not 
support the learning by discovery hypothesis, nor does it replicate the 
Bunderson et al., (1970b) findings of no significant differences between 
rule and no-rule treatments. This lack of replication may be due to 
the fact that the transfer test used in this study had twice as many 
items as the test used in the earlier study. In the section in which 
the learning by discovery issue was discussed it was suggested that 
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presenting objectives would allow Ss to organize relevant information so 
as to facilitate transfer to a new task. Even though there was a weak 
trend in favor of the objective groups, it seems that precisely stated 
rules have a greater effect on transfer retrieval than objectives. The 
weak objective effect may have been due to the fact that the objectives 
orly specified that transfer retrieval would be required to solve new 
problems using previously demonstrated relationships. Objectives did 
have a significant effect on transfer retrieval (number of examples 6 
test item response latency) where the criterion items were constructed 

according to the original objectives. 

The significant rule effect on n\imber of examples found in this 

study replicates the results of previous studies (Bunderson et al. , 1970b), 
and the significant objective effect supports the hypothesis that 
objectives would reduce the number of examples required to meet criterion 
by helping the to focus on the relevant stimuli in the displays. The 
frequency distributions (Figure 2) show that the presentation of rules 
enabled most Ss to learn the science in a minimum number (10) of trials 
and therefore with nearly zero errors. Objectives had a similar but less 
pronounced effect. Since the rule treatments brought such a high percent- 
age of Ss to perfect performance in terms of the number of examples 
required, the full impact of these treatments using number of examples as 
criterion is indeterminate. However, the within group variance was not 
similarly restricted in the latency criterion measures. 

The hypothesis that presentation of rules would significantly 
reduce the amount of time required to learn the task was supported by 
significant rule effects on all three latency measures. Although a floor 
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effect was not observed in the rule groups using latency measures as criteria, 
the rule treatments seemed to effect a substantial decrease in latency within 

group variance. 

The presentation of objectives did not have the hypothesized effect 
of reducing the total time required to complete the task. This result would 
seem to contradict the argument that objectives have a focusing effect if it 
were not for the reduction in the number of examples required by the objec- 
tive groups. A comparison of the component latency measures, display and 
test-item-response latency, revealed that objectives either increased or 
had no effect on display latency but significantly reduced test -item-response 
latency. Apparently the presentation of objectives affected the efficiency 
and effectiveness of the S's information processing and thereby facilitated 

his performance on the criterion test items. 

The hypothesis that objectives and/or rules would reduce the 
relationship between certain cognitive abilities and task performance was only 
partially supported. Apparently the assumption that task performance would be 
related to associative memory was not valid. Therefore, a reduction in the 
requirement for associative memory for £s who received rules and/or objectives 
could not be detected. There are at least two possible explanations for the 
lack of relationship between task performance and memory. First, the task may 
have required only a certain minimum level of memory ability possessed by all 
Ss, and second, the task may have required some other type of memory ability 
such as memory span instead of associative memory. The interaction between 
reasoning factor scores and the treatments with number of examples as crite- 
rion only approached significance. The difference between the regression 



line slopes for the Rule-Example and Example-Only groups was similar to that 
found in earlier studies (Bunderson et al., 1970b), but the Objective-Example- 
treatment did not effect a r(='duction in the slope with number of examples as 
criterion. However, the presentation of objectives and/or rules did signi- 
ficantly reduce the relationship between reasoning factor scores and test- 
item-response latency. There was a similar significant reduction in the 
relationships between Letter Sets Test scores and total latency for £s who 
received rules and/or objectives. Why the treatments interacted with rea- 
soning abilities using test-item-response latency and total latency as cri- 
teria and did not interact significantly with display latency and number of 
examples as criteria is not clear. Apparently reasoning abilities are more 
crucial during the stages of the task when Ss respond to the criterion test 
items, and therefore the objectives and/or rule treatments compensation for 
these abilities is more evident during those stages. No ability by treat- 
ment interaction hypotheses were formulated concerning the post, retention 
and transfer criterion measures. It was not assumed that the same abilities 
which would be required to learn the task would also be required to perform 
the terminal behavior of the task. Therefore, the regression analyses using 
post, retention, and transfer test scores were only conducted for exploratory 
purposes. However, no significant ability by treatment interactions were 

found using the post-task criterion measures. 

The hypothesis that objective effects would be greater between the 
Example-Only and Objective groups than between the Rule and Rule-Objective 
groups was only supported significantly by the interaction found with test- 
item- response latency as criterion. However, an examination of the means 
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for the other criterion measures shows that the corresponding differences 
between the means of the other criteria are consistent with the hypothesis. 
Thus it is impossible to make broad general statements about the effect of 
objectives on the learning process without taking into account the other 

stimulus properties of the task. 

The purpose of this study was to investigate what effects the pre- 
sentation of behavioral objectives would have on the learning process. On 
the basis of the results of this study it was concluded that objectives have 
orienting and organizing effects which dispose students to attend to and 
organize relevant information and thus facilitate performance on criterion 
test items constructed in accordance with the objectives. The organizing 
effect of objectives also compensates for reasoning abilities required to 
respond accurately to the test items based on the objectives. Objectives 
have a greater effect when added to a task without other orienting or organ- 
izing stimuli than when they are added to a task with other organizing stim- 
uli such as rules. Rules have similar but. somewhat more pronounced effects 
than objectives. Contrary to the learning by discovery hypothesis, the pre- 
sentation of rules facilitates performance on a transfer task where higher 
order rules must be inferred from examples. However, the presentation of 
objectives for one task does not necessarily facilitate performance on a 
transfer task with different, though similar, objectives. Objectives and 
rules seem to perform different functions since there is an apparent addi- 

tive effect when they are presented together. 

The data of the present study have several methodological impli- 
cations. One of the major implications seems to be that latnecy data may 
be of greater value than has been previously supposed. Even though total 
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latency is probably of greater practical importance in terms of the differ- 
ential costs required to use various instructional treatments, it may be of 
greater psychological importance to. examine latency in terms of its different 

components as was done in the present study. 

Second, even though none of the ability by treatment interactions 

found in this study are of great practical importance in terms of individual- 
izing instruction by using ability profiles, they are of theoretical impor- 
tance since they facilitate analysis of the.effeots various treatments have 
on the learning process of Ss with different ability strengths. Thus, the 
results of this study further support Dunham and Bunderson.’s (1969) conten- 
tion that it is possible to vary the nature of the cognitive processes in 

learning by manipulating a task variable. 

Third, a comparison of the difference between the. squared multiple 

correlations of the criterion measures and the task relevant versus the pub- 
lished ability tests suggests that the use of task relevant tests -n abil.ity 
by treatment interaction studies should receive continued consideration. 
Through further revision and testing of the task relevant tests it should be 
possible to increase their reliability and multiple correlation with criteria. 
It is of further interest to note that the predictive effectiveness of task 
specific tests may vary depending upon the type of instructional treatment 
the sample receives. (For the Rule-Example group the published tests corre- 
lated higher with criterion measures than did the task relevant tests.) 

In conclusion, the reader is cautioned against over-generalization 
of the results of this study. It should be remembered that the learning task 
was a highly symbolic imaginary science presented by computer-assisted instruc 
tion, and that Ss were required to participate as a class requirement. Thus 
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it is dangerous to generalize to highly different tasks, methods of presen- 
tation, and/or populations of Ss. Nevertheless, the implications of this 
study and its methodology are such that it would be valuable to attempt to 
replicate the major findings using other tasks and populations. Future 
research on objectives needs to be conducted to investigate what effects 
QjjjgQ-tives would have on terminal behavior criterion measures if the number 
of examples and amount of time allowed for each treatment is controlled. 
Since £s in the no-objective groups of the present study could have inferred 
the objectives from the criterion test items presented after each example, 
•the full effect of objectives could not be determined. Therefore, future 
research on objectives should also examine the differential effects of ver- 
bally stated objectives and test items based on objectives. 
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SUMMARY 



The purpose of this study was to investigate what effects the 
availability of behaviorally stated objectives would have on the learning 
process. It was hypothesized that objectives would serve as orienting stim- 
uli which dispose the student to attend to, process, and organize relevant 
aspects of displayed information in accordance with the stated objectives. 
Therefore, the presentation of objectives was expected to reduce the num- 
ber of examples and amount of time required to learn the task, facilitate 
performance on transfer retrieval criterion measures, and reduce the require- 
ments for memory and reasoning abilities. 

The learning task consisted of a hierarchical imaginary science 
called the Science of Xenograde Systems which was presented by an IBM 1500/ 
1800 Computer- Assisted Instruction System to 130 introductory educational 
psychology and science education students . 

Before learning the task, all were given a battery of six cogni- 
tive ability tests comprised of three existing published tests and three task- 
relevant tests developed for this study. The tests were selected to measure 
abilities postulated to have specific relationships to task performance. 

The £s were randomly assigned to an Example-Only group, an Objec- 
tive-Example group, a Rule-Example group, and an Objective-Rule-Example group. 
All Ss were required to meet a minimum criterion performance at each level 

of the task before proceeding to the next level. 

A significant rule effect was found on the number of examples and 
the total time required to complete the task with the rule groups needing less 




53 



time and fewer examples than the no-rule groups. Oliectives sirni i -c intly 
reduced the number of examples required to learn '.he tas.%, but they did not 
reduce total latency. An analysis of the component latency meai-ures revealed 
that objectives either increased or had no effect on display latency but sig- 
nificantly reduced test-item-response latency. Apparently the objective 
treatments affected the efficiency anci efiectiveness r>f the ^ s j.n. c: r.at j.on 
processing and thereby facilitated his periormance on the criterion-test 
items based on the objectives. 

No significant differences were round betvceen treatments on t.ne 
post or retention tests , but a significant rule effect in favor of the rule 



groups was found on the transfer test. 

The battery of cognitive tests was factor analyzed, and a two- 
factor varimax solution was obtained yielding the factors of reasoning and 
associative memory. Regression analyses of the individual ability test 
scores, factor scores, and the criterion measures were conducted. Signifi- 
cant ability by treatment interactions were obtained using test— item— response 
latency as criterion and reasoning factor scores, plus individual reasoning 
tests as covariables. Reasoning had a high negative relationship to test- 
item- response latency for in the Example-Only group, but this relationship 
was significantly smaller for Ss in the remaining groups. Therefore, the 
presentc jion of objectives and/or rules effected a reduction in the require- 
ment for reasoning ability. 

On the basis of the results of this study it was concluded that 
objectives have orienting and organizing effects which dispose students to 
-to, process, and structure relevant infonnation in accordance with 




the given objectives. 
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APPENDIX A 

Sub -Objectives and Rules 
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APPENDIX A 



Sub-Objectives for the Science of Xenograde Systems 



1. Given that F.F. = 1, and the values of ACS and the previous distance 
predict the value of the next distance. 

2. Given that ACS = 1. and the values of F.F. and the previous distance 
predict the value of the next distance. 

3. Given the values of F.F. , ACS c^nd the previous distance, predict the 
value of the next distance. 



4. 



5. 



6 . 



7. 



8 . 



Given the previous values of ACN and ACS, and that no blip has occurred 
predict the next values of ACN and ACS. 

Given the value of the time and that a blip has occured, predict the 
blip time and the value of the distance at that tune. 

Given the previous values of ACN and ACS, and that the blip time is 
even, predict the next values of ACN and ACS. 



Given the previous values of ACN and ACS, and that the blip time is 
odd, predict the next values of ACN and ACS. 



Given the previous value of ACS, 
ACN was zero on the previous line 



that the blip time is even, and that 
, predict the next values of ACN and ACS. 



9. 



Given the values of F.F. . ACS, and that a blip has occurred, predict the 
next distance . 



10 . 



Given the distance at time zero, predict the maximum value the distance 
will reach. 
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Rules for the Science of Xenograde Systems 

1. If F.F. = 1, the decrease in distance between each time is equal to ACS. 

2. If ACS = 1» the decrease in distance between each time is equal to F.F. 

3. The decrease in distance between each time is equal to the value of 
F.F. X ACS. 

4. ACN and ACS cannot change unless a blip occurs. 

5. VIhen the distance beeves zero a blip is recorded whose value is equal 
to the value of the time. 

6. When the blip time is even . ACai decreases by one while ^ increase s by one 

# 

7. When the blip time is oM, ^ increases by one while ^ decre^e ^ by one. 

8. If the blip time is even and ACN was zero on the previous line, ACN and 
ACS do not change. 

9. After a blip occurs, the distance begins to increase each time by the 
value of F.F. x ACS . 

10. After a blip, the distance increases to its value at time zero then begins 
to decrease again. 
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Instruction Booklet with Xenograde Table 
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APPENDIX B 

Student Instruction Booklet 

The instructional program concerns an imaginary science called 
the Science of Xenograde Systems. A Xenograde System consists of a 
nucleus with an orbiting satellite. The satellite is composed of small 
particles called alphons which may also reside in the nucleus. Under 
certain conditions ^ a satellite may collide with the nucleus. When such 
a collision occurs, a "blip" is said to have occurred, and the satellite 
may exchange alphons with the nucleus. The science deals with the laws 
by which the activity of satellites and alphons may be predicted. 

The following diagram is one way of conceptualizing a 
Xenograde System: 




ORBIT 
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Justifications 

Your participation in the study of Xenograde Systems will enable 

the research staff of this laboratory to study how people learn a science 

and how they form an-* test hypotheses. 

The time you spend will not give you an encyclopedia of facts 

useful outside this course, but it may improve your skills of observation, 
inference, prediction, formulating hypotheses, controlling and manipulating 
variables, interpreting data, formulating models, and a better way of approach- 
ing scientific problems. The study you are about to undertake has the chal- 
lenge of a complex game and should be interesting in its own right. 

The interaction with the materials in this study will give you some 
idea of the potential of computer-assisted instruction in simulation of a 
science and testing. Uter you may want to sample some demonstration programs 
showing other uses of computer-assisted instruction. 
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Insti'uctions for Reading "the Displays 



In taking this course, you will need to be able to read a tabular 
display on the CRT which records the activity of the particles making up a 

Xenograde System. 

Figure 1 is a sample display. 



FF = 2 

System 

Time 



0 

1 

2 

3 

4 

5 

6 



ACN 

2 

2 

2 

2 

1 

1 

1 



Blip 

Time 



4 



Satellite 

Distance 



ACS 



24 ^ 

18 3 

12 3 

6 3 

0 

8 4 

16 4 



Figure 1. Sample display of a Xenograde table. 

The symbols stand for the following. 

F.F.- Force field - Physically this can be thought of as an area 
in space, which if entered by an Xenograde system, will exert certain pre- 
dictable effects on the system. The strength of the force field can be 
measrued and given nmterical values. The effect of the force field on the 

Xenograde System is based on the strength of the force field. 

Time- This column serves as a clock which provides a basis for 
presenting the state of the system at small sequential intervals of time. 
It is increased by a value of 1 (one) with each reading. Notice that time 



always starts at time 0 (zero). 
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ACM- Alphon Count of the Muoleus. As the name suggests, the numer 
ical values in the oolvmn under ACM refer to the number of alphons that are 
located in the nucleus at any given time. For example, in the figure the 
number of alphons in the nucleus at time 2 is 2 while the nu«d>er of alphons 

on the nucleus at time 6 is 1. 

BLIP time- In the column under this heading are recorded the values 
of the time clock when a blip occurs, that is when a satellite collides with 
the nucleus. In Figure 2 you wiU notice that such a coUision occurred at 

time 4. 

satellite distance- The values recorded in the column under this 
heading refer to the number of units of distance between the satellite and 
the nucleus. Ft^om Figure 2 you wiU notice that the satellite is 24 units 
from the nucleus at time 0 while it is only 6 units from the nucleus at time 3. 

Alphon count of the Satellite. The values recorded in the 
column under this heading refer to the number of alphons which make up the 
satellite at any given time. For example, in the Figure, the number of alphon 
in the satellite at time 2 is 3 while there are 4 alphons in the satellite at 

time 5. 

•: - A series of three dots in any column refer to a series of values 
that have'been skipped. For example, if the time column starts with three 
dots followed by the number 24. then all the values from time 0 to time 24 

have been skipped. 
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Instructions for Group 6 (X) 



‘ J 



Follow these instructions in taking the coiirse. 

1. After the proctor signs you on the terminal you will be instructed in how 
to use the terminal and given time to practice typing in numbers and 
correcting errors. 

2. When you begin the course a Xenograde display table will appear on the 
Cathode Ray Tube (CRT). Your task will be to study each table as it 

is presented and try to discover a- rule which determines how the values 

in the tables change. 

3. After you have studied the table,, type the letter c to continue. 

4. Next you will then be given a series of 3 test items. These test items 
will consist of partial tables with missing values represented by a 
shaded box. You will be asked to predict the missing values by using 
the rule you have discovered. After typing in your answer and performing 
the ENTER Function, you will automatically be given the next item. After 
taking the three test items, you will be told how many you answered 

correctly. 

5. If you miss more than one out of three test items, you will be shown 
another table based on the same rule followed by another series of three 
test items. You may receive up to five tables followed by test items 

for each rule. 

6. If you answer at least 2 out of 3 test items correctly, a new table will 
be displayed based on the next rule. 



r*l 

/ / 
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7. You will follow the above procedures repeatedly until the 10 rules of 
the science have been learned. 

8. After learning all the rules of the science, you will take a posttest. 

The posttest will assess your ability to predic': entries in a table of 
Xenograde readings line by line given the initial conditions. Since the 
scores you make in learning this course will not affect your grade, but 
will be used to answer research questions in education, we would appre- 
ciate it very much if you wou^^refrain from discussing the details of 
the science and posttest with fellow class mates who have not yet taken 
the course. Prior knowledge of the details of the course may confound 
the results and make the time you have spent in vain. 

Please make no notes of any of the instructional material. Paper 
and pencil are not allowed to be used during any of the instruction at 
the computer terminal. One goal of this research is to investigate 
your ability to remember without using notes or any reference materials. 

PLEASE NOTE: If you run into difficulty, it will be very helpful 

for you to refer back to this booklet. Try to relate tne numbers in 
the tables to the physical diagiw and the explanation found on the 
first psge of this booklet. 
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APPENDIX C 



Sas^le Intra-Task Test 
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APPENDIX C 



Sample Intra-Task Test 



V 

h 



After studying each example in the task, the Ss respond to a three 
item constructed response test. The following are sample items taken from 
ont. of these tests . 





1 . 



F.F. = 3 



SYSTEM 

TIME 



0 

1 



ACN 



BLIP 

TIME 



SATELLITE 

DISTANCE 

45 

9 



What is the value of the distance at time 1? 



ACS 

5 

5 




2 . 



F.F. = 4 



SYSTEM BLIP 

TIME ACN TIME 



0 

1 



SATELLITE 

DISTANCE 

32 

9 



What is the value of the distance at time 1? 



ACS 

2 

2 




3. 



F.F. = 4 



SYSTEM 

TIME ACN 



0 

1 



BLIP 

TIME 



SATELLITE 

DISTANCE 

24 

9 



What is the value of the distance at time 1? 



ACS 

3 

3 
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APPENDIX D 

Post and Retention Test Instructions 
and Sample Items 




APPENDIX D 

Post and Retention Test Instructions 
and Sample Items 

The purpose of this test is to assess your ability to use all the 
rules you have learned to predict entries in Xenograde Tables given cer- 
tain previous conditions. 

For each item you will be asked to fill in an entire line of a 
Xenograde Table. Thus you will make four predictions for each item. Write 
your predictions in the spaces provided on the last line of each table. 

In some of the items, there may be cases where no entry should 
be made in a column. When this occurs leave the appropriate space blank. 
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Sample Items from Posttest 



1. 


F.F. = 2 












SYSTEM 




BLIP 


SATELLITE 


ACS 




TIME 


ACN 


TIME 


DISTANCE 




0 


3 




40 


2 




1 


— 




" " 


~ 


2. 


F.F. = 1 












SYSTEM 




BLIP 


SATELLITE 


ACS 




TIME 


ACN 


TIME 


DISTANCE 




0 


• 


• 


28 


• 




1 


4 




25 


3 




2 


— 


— — 






3. 


F.F. = 3 












SYSTEM 




BLIP 


SATELLITE 


ACS 

• 




TIME 


ACN 

• 


TIME 

• 


DISTANCE 

• 




• 

• 

14 


• 

• 

0 


• 

• 


• 

• 

18 


• 

• 

3 

Q 




15 


0 




9 


O 




16 


— 


— 






4. 


F.F. = 2 












SYSTEM 




BLIP 


SATELLITE 


ACS 




TIME 


ACN 


TIME 


DISTANCE 




0 






32 






• 


• 


• 


• 


• 




• 


• 


• 


• 


• 




• 

4 


• 

0 


• 

4 


• 

0 


5 
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and Sample Items 
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Transfer Test Instructions and Sample Items 



Instructions for Transfer Task 



For the transfer task you wiU be given two Xenograde tables which 
,iU serve S exLples for three new rules of the Science. Your task wxU be 
ft Q-htitlv these tables in order to discover the additional rules* 

■ ^ men yS feel you have discovered the rules, go to the test itens 

,here you have upon alphon activity. 

3'. ma? ?hrM^^Sst^«‘*im'bf if the distance incren«nt would 
take the satellite past its original orbit. 



FF = >2 
System 
Time 

0 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 



Example 1 



ACN 

2 

2 

3 

3 

3 

3 

3 

3 

4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 

4 

5 






8 



20 



Satellite 
Distance ACS 

12 3 

6 3 

0 2 

4 2 

8 2 

12 2 

8 2 

4 2 

0 1 

2 1 

4 1 

6 1 

8 1 

10 1 

12 1 

10 1 

8 1 

6 1 

4 1 

2 1 

0 0 



The Satellite disappeared at time 20. 



FF = 2 
System 
Time 
“0 
1 
2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 
27 



Example 



ACN 

“T" 

2 

1 

1 

1 

2 

2 

2 

2 

3 

3 

3 

3 

3 

3 

4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 

4 

5 



BiiE 



Satellite 

Distance 

12 



15 



27 



6 

0 

8 

8 

0 

6 

12 

6 

0 

4 

8 

12 

8 

4 

0 

2 

4 

6 

8 

10 

12 

10 

8 

6 

4 

2 

0 



ACS 



3 

4 
4 
4 
3 
3 
3 
3 
2 
2 
2 
2 
2 
2 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
0 



The Satellite disappeared at time 27, 






Sample Transfer Test Items 



SYSTEM 

TIME ACN 



BLIP 

TIME 



SATELLITE 

DISTANCE 



• 

57 

58 

59 



8 

8 

■> 59 



• 

12 

6 

0 



Will the satellite disappear at time 59? .. _ 

(yes or no) 




2. 


F.F. = -5 










SYSTEM 

TIME 


ACN 


BLIP 

TIME 


SATELLITE 

DISTANCE 




0 






25 



{ 



5 

6 

7 

8 



• 

2 

2 

2 

2 



0 

10 

20 

0 



At time 8 the value of the distance is 



'i 




F.F. = -3 








SYSTEM 

TIME 


ACN 


BLIP 

TIME 


SATELLITE 

DISTANCE 


• 

• 

• 

13 

14 

15 


• 

• 

• 

3 

3 

9 

• 


• 

• 

• 

15 


• 

• 

• 

24 

12 

0 


At time 15 


the value 


of ACN = 


and ACS = 




APPENDIX F 



Task Relevant Test Instructions and Sample Items 
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memory of number series test 

Instructions and Sample Items 



TV,i*s i «3 a test of your ability to learn combinations of letters 

Your task will be to study a page of 15 pairings 

iown the number series which goes with each letter. 

Hare is a nractice list of letter-number series pairs. Study 
this list until you are asked to go to the next page. one minu e 



F - 9 0 9 0 
K - 2 2 2 6 
R - 1 2 3 4 
M - 4 5 4 3 

G - 8 8 8 8 



DO NOT TUBN THIS PAGE UNTIL ASKED TO DO SO 
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Practice Test Page 



The correct number series which corresponds to the letter F has 
been entered. Write all the other number series which correspond to the 
given letters that you can remember. 



F - 9 0 9 0 

K - 

R - 



M - 



G - 

Your score will be the number marked correctly. Even if you 
are not sure of the correct answer to a question, it will be to your advan< 

tage to guess. 

There are two pages to the test. The first of these is a memory 
page which you are to study for 4 minutes. 

The second is a test page on which you are to write the numbers 
series that go with the letters. You will have two minutes to write. 



DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO 
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3X—C0LUMN NlWBtJR SERIES TEST 
Instructions and Sample Items 



Each problem in this test has four sets of numbers ganged in two 
columns each. The first three sets demonstr. te a 

Cionship between the two columns. Tour task will be to . 

and then to use it to predict a value in the second column of the fo^h 
set A different rule will be used for each problem. You are to 

answer^wTiting the predicted value on the answer line provided in 

the extreme right hand column. 



Please study the following example carefully, . 



Example A : 




In Example A the correct answer is 13. This answer is marked for 
vou in the space provided in the right hand column. The answer - arrived 
at by the follow!^ rule: Multiply the firet two numbers ’ 

then divide their product by the third number in column a. Column b will 
decrease by the quotient of this division. 



Now try to derive the rule amd the answer for Example B 



Example B : 



7 

6 

5 

4 



65 

52 

43 

34 



9 

6 

3 

0 



45 

3C 

27 

24 



16 

12 

8 

4 



b 

58 

30 

18 

6 



a 

10 

8 

6 

4 



b 

58 



In B, the values in column b decrease once by an ^unt 

.nual to the th^ first two values in column a. Then colu^ b decreases 

5“e by III si of the last two values in column a. ^The «^ct answer is 30, 
iotice that the answer required for Example B is in the third row. while the 
inswer for Example A is in the second row. 

Some of the remaining items in this test will be easier than the 
sxamples above, while others wiU be morr difficult. You e^ect to 

Find*^any kind of rule or operation governing the relationship between the 
Lo colLns. You will have 8 minutes in which to complete the test. 

Tv-> MrtT TiiPN THIS PAGE UNTIL ASKED TO DO SO 



CtC » 
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TOTE MOBILE TEST 
Instructions and Sample Items 



The accoapanying diagram represents the course for a TOTE MOBILE. 
The TOTE MOPILE may travel up and down or from side to side, but i^y no 
SLel diagonally. Each squ^ has a label corresponding to the intersection 
of the numbered columns and lettered rows. As examples, note squares C6, D4 
and D8 in the diagram. The vertical lines are labeled with numbers in paren- 
theses while the horizontal lines are labeled with small letters. 

REFER TO THE ACCOMPANYING DIAGRAM FOR ALL THE FOLLOWING PROBLEMS. 

Below is a chart showing the course of the TOTE MOBILE: 



Start 



Location D4 PS D6 D7 D8 



rhus, the TOTE MOBILE traveled from D4 to the right for * 

Your task will be to predict the final destination (END SQUARE) of the 

rOTE MOBILE based on the following given conditions. ^ 

Condition I. Tote Mobiles begin traveling to the right, 
rendition II. Tote Mobiles travel ona square for each cup of fuel. 



Look at these two sample items: 
1) FUEL = 5 



Start 



Location G1 G2 G3 G4 G5 G6 



END SQUARE = G6 



2) FUEL = 6 



Start 



Location H4 H5 H6 H7 H8 H9 HIO — - — - 

END SQUARE = HIO 

In item 1 the TOTE MOBILE started in square G1 and traveled to the 
right (condition I) for 5 squares (condition II). Thus its final destination 
(G6) is entered in the space provided after END SQUaRE =. 
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In item 2 the TOTE MOBILE traveled six squares to the right since 

“ ^ “r :^y'u:^he e»pty spaces provided to Keep tracK °^he MOmE-s 

course, however only the value recorded in the blank provided for the END 

SQUARE the END SQUARE for the following TOTE MOBILES 

based on the above conditions and the following condition: 

Condition III. If a border (line 11 , 1 . a or k) is reached the TOTE MOBILE 

direction. 



3) FUEL = 4 



Start 



Location 




END SQUARE = 



4) FUEL = 4 



Start 



Location 




C7 

END SQUARE = 



For item number 3 you should have entered in the space provided 
.fter END fol^ w-n 3 min- 
utes remain. If you have any questions ask them now. 



DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO 
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Tote Mobile Test Diagram 




REFER TO THIS DIAGRAM TO SOLVE THE TOTE MOBILE PROBLEMS. 



on 



APPENDIX G 



Linear Regression Models 



APPENDIX G 



Linear Regression Models 

In order to test the ability by treatment interaction hypothe- 
ses made in this study, analyses of linear regression mciels were conducted 
A computer program prepared by Jennings (1968) entitled Program Linear 

was used for the analyses. 

The models which were discussed in the section entitled Analysis 
of Ability by Treatment Interactions are defined mathematically as follows: 



Model 1 

Y = a^X + a20 + a3R + a^RO + bj(X*A) + b2(0*A) + b3(R*A) + b^CRO-'A) + 

Where 

Y = vector containing the criterion score for each S^. 

X = vector in which the element is a 1 if the corresponding element 

in Y is a score for a S in the Example-Only group , zero otherwise . 

0 = vector in which the element is a 1 if the corresponding element 

in Y is a score for a S in the Objective -Example group, zero 

otherwise . 

R = vector in which the element is a 1 if the corresponding 

in Y is a ‘score for a S in the Rule-Example group, zero otherwise. 

RO = vector in which the element is a 1 if the corresponding element 

in Y is a score for a S in the Objective-Rule-Example group , 

zero otherwise . 

* 

A = ability score of a £ whose corresponding criterion score is an 
element of vector Y. 

X*A 0*A. R*A, and RO*A are direct product vectors which are obtained 
by multiplying each element of the first vector by its correspond- 
ing elemL? in the second vector. Thus X*A is a 

ability scores for Ss in the Example-Only group, and zero otherwise. 



82 



a,a,a,a,b,b,bjbj^= unknown coefficients, or weights, 
^ associated**witA their corresponding vectors. These weights are 
estiiiBted by a least-squares procedure. 

E = residual vector in which elements are the differences between 
the observed and estimated values in the criterion vector Y. 

The above and following models assume that for each unit of change 
in ability score there is an equal change on the criterion. This assump- 
tion is commonly referred xo as the assumpliou of linearity. 

In effect, the estimated values for bi, b 2 , ba, and b 4 are the 
slopes of the treatment group regression lines. Thus, the above model 

allows the four slopes to be different. 

To test the null-hypothesis that the slopes are all parallel, the 
following restrictions are imposed on the slope coefficients of Model 1: 

bj = b2 = b 3 = b 4 = bo (a common weight) 

giving the following restricted model: 

Model 2 

Y = ajX + 320 + a^R + a4R0 + boA + E 2 

where the vectors are defined as under Model 1 above. 

To test the null-hypothesis that the regression lines' slopes for 
the Objective -Example group, the Rule-Example group, and the Objective- 
Rule-Example groups are all mutually parallel, and the slope of the 
Example-Only group regression line is allowed to be different, the fol- 
lowing restrictions are imposed on the coefficients of Model 1: 



Q P. 
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h =b =b =Cofa common weight) 
^2 3 4 

giving the following restricted model; 



Model 3 



Y = 






ajR + a 4 R 0 + bi(X*A) + coZ 



+ Ej 



Where 

Z = (0*A) + (R*A) + (RO*A) = vector containing 
Ss in the 0bjecxive-Exair4pie, Rule-Example* 
Example groups, and zero otherwise. 



ability scores for 
Ohj<artive-Rul»- 



Q7 



APPENDIX H 



Correlation Matrices 



APPENDIX H 



ERIC 



Table 23 



Intercorrelation of 



Criterion Variables for All Groups 



a ,b 



Variable Number 


1 


2 


3 


4 


5 


6 


7 


1. 


Post 


100 


82** 


60** 


-35** 


-2 /•• '• 


-24** 


-28** 


2. 


Retention 




100 


60** 


-35** 


-25** 






3. 


Transfer 






100 


-43** 


-31** 


-35** 


-36** 


4. 


No. of Examples 








100 


60** 


7 3** 


73** 


5. 


Display Latency 










100 


70** 


92** 


6. 


Test- Item-Response 
Latency 












100 


92** 


7. 


Total Latency 














100 



^Decimal points omitted. 

^n = 127 for display and total latency; 
n = 130 for all other criteria. 



*£ < .05 

< .01 
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Table 24 

Correlations of Criterion Variables with Cognitive Ability Covariables^ 




Memory of Number 


Series 


(KA) 


31 


31 


42* 


-36* 


-21 


-28 


-27 


First 6 Last Names (HA) 




27 


11 


23 


-27 


-26 


-25 


-26 


Bi-Column Number 


Series 


(I) 


14 


19 


12 


-21 


-21 


-26 


-21 


Letter Sets u) 






■^9 


23 


11 


«4 5** 


-45 


-57** 


-54** 


Tote Mobile (R) 






43’ 


33 


34 


-46*«* 




-53 — 


-51** 


Ship Destination 


(R) 




35' 


63** 


43* 


-52** 


-36* 


-53** 


-46** 


Reasoning Factor 






39‘ 


46** 


32 


-52** 


- 47 ** 


-59** 


-54** 


Memory Factor 






27 


11 


27 


-26 


-22 


-22 


-23 










Object ive 


-Example Group® 









Memory of Number Series (HA) 


10 


-01 


06 


-15 


-22 


-15 




First & Last Names (MA) 


05 


-02 


00 


-16 


-34 


06 


-19 


Bi-Column Number Series (I) 


C3 


17 


21 


-38* 


-22 


-32 


-29 


Letter Sets (I) 


15 


21 


11 


-17 


00 


-08 


-04 


Tote Mobile (R) 


26 


16 


35* 


-15 


-19 


-03 


-14 


Ship Destination (R) 


4 5*; 


: 42 * 


4 0* 


-42* 


-33 


-31 


-35* 


Reasoning Factor 


36* 


37* 


40* 


-40* 


-24 


-28 


-29 


Memory Factor 


01 


-09 


-03 


-11 


-28 


01 


-17 


Rule -Example Group^ 



Memory of Number Series (MA) 


18 


04 


07 


-34 


-28 


-28 


-29 


First 5 Last Names (MA) 


07 


02 


02 


-22 


-08 


-10 


-09 


Bi-Column Number Series (I) 


20 


20 


08 


-38* 


-39* 


~50** 


-47** 


Letter Sets (I) 


17 


10 


22 


-26 


-20 


-19 


-21 


Tote Mobile (R) 


4 5** 


38* 


54** 


-31 


-51** 


-37* 


-46** 


Ship Destination (R) 


32* 


22 


35* 


-42* 


-53** 


-44* 


-51** 


Reasoning Factor 


37* 


29 


41* 


-44* 


-53** 


-47** 


-52** 


Memory Factor 


10 


00 


-02 


-20 


-12 


-14 


-14 



Objective-Rule-Example Group® 



Memory of Number Series 


(MA) 


02 


08 


32* 


-14 


-27 


-34 


-32 


First 6 I>ast Names (MA) 




11 


23 


41* 


-24 


-10 


-32 


-20 


Bi -Column Number Series 


(I) 


24 


38* 


09 


22 


-22 


-1:^ 


■> 

A*- 


Letter Sets (I) 




16 


31 


04 


-30 


-64** 


- 54 ** 


-65** 


Tote Mobile (R) 




37* 


50** 


35* 


-44* 


-54** 


-48** 


-56** 


Ship Destination (R) 




45** 


38* 


07 


00 


-26 


-18 


-25 


Reasoning Factor 




41* 


51** 


15 


-16 


-54** 


-4 2* 


- 54** 


Memory Factor 




02 


12 


40* 


-23 


-21 


-3'^* 


-2'^ 



^Decimal points emitted. 

^n = 30 for displa^♦ and total latency, H other criteria. 

^nT = 3.? for display and total latency, H - other criteria, 

^n = 32 
®n = 33 
**‘R ^ *05 
**£ < .01 
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